Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis

Background We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs. Methods and findings We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations. Conclusions We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Abstract Background We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.

Methods and findings
We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment.
The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.

Conclusions
We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with

Author summary
Why was this study done?
• Registration of a clinical trial is crucial to prespecify outcomes, deter publication bias, and avoid unnecessary duplication of clinical research.
• The rate of prematurely discontinued trials (primarily due to preventable reasons) and nonpublished trials was high in a study conducted 10 years ago.
• An assessment, providing robust data on the proportion of randomized trials that are registered, and an update to explore if the proportion of discontinued and nonpublished trials has changed, was warranted.
What did the researchers do and find?
• We assessed whether 326 study protocols of randomized clinical trials that received ethical approval in 2012 (in Switzerland, the United Kingdom, Germany, and Canada) were registered, completed, and published.
• While results are available on a trial registry for 84% (150/179) of industry-sponsored trials, this is only the case for 16% (23/147) of investigator-sponsored trials.

What do these findings mean?
• This metaresearch study revealed that discontinuation of randomized trials is still common, contributing considerably to research waste as the most common reasons for discontinuation appear preventable.
• In terms of making trial results available, the situation has improved during the last 10 years.
• Trialists should be encouraged to demonstrate feasibility before embarking on definitive trials, and further efforts are required to improve reporting of trial results in registries for investigator-sponsored trials.

Introduction
Rigorously planned and conducted randomized clinical trials (RCTs) are critical to inform the effectiveness and safety of healthcare interventions [1,2]. Clinical trial registries were implemented in the early 2000s to avoid unnecessary duplication of research and to estimate and deter publication bias. In 2005, the International Committee of Journal Editors proclaimed prospective trial registration (i.e., registration before enrolling the first participant) as a requirement for publication [3,4]. Shortly thereafter, laws in different regions (e.g., European Union and North America) required trials to be registered [5,6] and making results available [7,8]. Several studies have explored the proportion of published RCTs that are registered [9][10][11][12]; however, such investigations used published RCTs-and not RCT protocols-as the denominator and thus were unable to assess publication bias. Trial discontinuation and nonpublication can constitute substantial research waste [13]. For example, if an RCT is discontinued due to slow participant recruitment before the planned sample size is reached, the trial is typically not sufficiently powered to answer the primary research question. The data, however, can still be useful in meta-analyses. Hence, it is crucial that all RCT results, including discontinued trials, are made available so that evidence is not lost and resource waste is minimized. We conducted an international metaresearch study of 1,017 RCT protocols approved between 2000 and 2003 that found 1 in 4 trials was prematurely discontinued, primarily due to poor recruitment [14]. Only 59% of approved trials were published at a median follow-up of 12 years, and premature discontinuation was associated with a lower likelihood of publication [14]. We acquired a new sample of RCT protocols, approved in 2012 by the same research ethics committees (RECs) plus an REC from the United Kingdom, to explore if trial completion and publication rates have changed. In addition, we aimed to investigate to what extent nonpublished RCTs can be found in trial registries, and the association between reporting quality of RCT protocols and trial discontinuation due to poor recruitment or nonpublication of results [15].

Methods
The present study is an associated project of the Adherence to SPIrit REcommendations (ASPIRE) study [15]. The ASPIRE study group is an international collaboration of researchers with a mandate to evaluate the completeness of RCT protocols before and after publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement [16]. Further ASPIRE substudies examine the use of patient-reported outcomes in RCT protocols, the reporting quality of RCT protocols with regulated versus nonregulated interventions, the planning of subgroup analyses in RCT protocols, and the use of routinely collected data in RCTs [15].

Study sample
The rationale and protocol for this study has been published [15]. In brief, we acquired 360 RCT protocols that were approved in 2012 by RECs located in Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St. Gallen, and Thurgau), the UK (the Bristol office of the UK National Research Ethics Service responsible for 19 RECs in the UK), Germany (Freiburg), and Canada (Hamilton; see S1 Text for details of participating RECs). We included RCTs in which participants were randomly assigned to different interventions (or an intervention and control group) to evaluate effects on health outcomes. We included all eligible RCT protocols that were available at participating RECs with the exception of Freiburg (Germany), Hamilton (Canada), and Zürich (Switzerland), where a random sample was selected (see study protocol for more details [15]). Studies labeled as pilot, feasibility, or phase 1 studies were excluded [15]. We also excluded duplicate protocols, and protocols for trials that were ongoing, had not started at the time of data collection, or were terminated but did not recruit any patients.
Reviewers determined, independently and in duplicate (for over 75% of included protocols), the reporting quality of all eligible RCT protocols by assessing the proportion of SPIRIT checklist items met [15][16][17].

Data collection
Reviewers determined whether each RCT was registered, prematurely discontinued (including reasons for discontinuation), and if trial results were published in a peer-reviewed journal or trial registry. In detail, we assessed if trials were registered by reviewing REC files and through a systematic search of the following registries and databases between March and September 2019: (1) the World Health Organization: International Clinical Trial Registry Platform (ICTRP) database; (2) US National Library of Medicine (ClinicalTrials.gov); (3) European Union Clinical Trial Registry (EUCTR); and (4) the ISRCTN registry. We also used the Google search engine to identify registration details. We classified an RCT as unregistered if we were not able to find any record or registration through our search. We used the following strategies to identify trials: (i) searching trial registration numbers (if reported in the protocol); (ii) full titles; (iii) short titles; (iii) study acronyms; and (iv) searching for the study population and intervention (with or without specifying the control group).
We extracted the trial status (i.e., completed, early discontinuation and why, or unclear), planned and achieved sample size, availability of study results, and reported links to full text publications. In February 2022, we rechecked the trial status and availability of study results for all included RCTs. We designated the trial status as unclear when an RCT was labeled as ongoing in the registry but the status had not been updated in the previous 2 years, unless the planned completion date was after February 2022. We contacted RECs and surveyed principal investigators for clarification when trial status was unclear (see S2 and S3 Texts for details). We classified RCTs as prematurely discontinued if they were specified as such in a trial registry, publication, or communication with a REC or trial author, or if the achieved sample size was <90% of the prespecified target sample size in the approved study protocol [14]. We conducted a systematic search of the following 3 electronic databases for full text publications corresponding to RCT protocols: PubMed, Google Scholar, and Scopus (see S4 Text for search strategies). For all corresponding full text publications identified, we extracted the planned and achieved sample size and, if applicable, the reason for premature trial discontinuation. All searches and data extraction were conducted in duplicate, and disagreements were resolved by discussion.

Analysis
Trial registration, publication, completion, and reasons for discontinuation are reported as frequencies and percentages with 95% confidence intervals (CIs), stratified by sponsorship (industry versus investigator) and country of study approval. We conducted univariable and multivariable logistic regression analyses with the following factors as dependent variables: (i) publication in a peer-reviewed journal; (ii) premature trial discontinuation due to poor recruitment; and (iii) discontinuation due to preventable reasons (considering discontinuation due to futility, benefit, harm, and external evidence as not preventable [18]; not prespecified in protocol paper [15]). These variables were selected because all RCTs should be published and discontinuation due to recruitment problems or preventable reasons should ideally be avoided (while other reasons for discontinuation, e.g., due to benefit or harm, might be in the interest of patients). We examined the following 7 independent variables in our models, hypothesizing that they might be associated with lower rates of discontinuation and nonpublication: (1) greater proportion of SPIRIT items reported in the protocol; (2) larger target sample size; (3) use of an active comparator versus placebo; (4) multicenter versus single-center study; (5) reporting of any recruitment projection versus not reporting; (6) industry-versus investigatorsponsored trials; and (7) support from a Clinical Trial Unit (CTU) or a Contract Research Organization (CRO) versus no support. The first variable was prespecified in our study protocol [15]. The other 6 variables were selected post hoc following our previous conducted analysis of trial protocols approved between 2000 and 2003 to facilitate comparison [14]. Variable 7 could only be included in a separate analysis in which we excluded UK protocols, as we were not able to extract from these protocols whether support from a CTU or CRO was provided. For all regression models, we calculated unadjusted and adjusted odds ratios (ORs) with 95% CIs. All analyses were conducted using Stata version 16.1 with p < 0.05 (2-sided) as the threshold for statistical significance (see S5 Text for statistical code).

Registration and protocol
Since PROSPERO does not allow the registration of systematic metaresearch studies that do not assess the effect of an intervention on a health outcome, we did not register this study. The protocol is published and publicly available [15].

Patient and public involvement and reporting
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research. This study is reported as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline [19,20].
The main reason for RCT discontinuation was poor recruitment (37%; 36/98) followed by stopping for futility (16%; 16/98; Table 3). The proportion of prematurely discontinued RCTs did not change compared to protocols approved a decade earlier (28% of protocols approved in 2000 to 2003 [14] versus 30% of those approved in 2012). Data comparing results from this study with the cohort from 2000 to 2003 [14] are presented in S4 Table. Switzerland had more unregistered trials (7.9%; 13/165), compared to the other countries (UK 4.5%, 4/89; Germany 2.7%, 1/37; Canada 2.9%, 1/35); otherwise, results were comparable in terms of discontinuation and nonpublication among the 4 included countries (S2 Table). Multivariable analyses indicated that RCTs with more complete protocol reporting according to SPIRIT guidelines [19,20] were less likely to be unpublished (OR 0.71 [in increments of 10% proportion of adherence]; 95% CI: 0.55 to 0.92; p = 0.009; Table 4). None of the assessed characteristics were found to be associated with discontinuation due to poor recruitment or discontinuation due to preventable reasons. A separate analysis excluding the sample from the UK found no association of CTU/CRO support with trial discontinuation or publication (see S5 Table).

Discussion
Our study found that most RCTs with ethics approval in 2012 (94%) were registered; however, 1 in 10 were registered retrospectively. Further, when restricted to unpublished RCTs, only 4 out of 5 trials were registered. RCT protocols with higher reporting quality, as indicated by greater adherence to SPIRIT guidelines [19,20], were more likely to be published in a peer- reviewed journal. Only 1 in 6 investigator-sponsored trials made results available in a trial registry. Approximately 1 in 3 RCTs were discontinued before the original planned sample size was reached, and 1 in 5 trials remained unpublished at 10 years follow-up. The results of discontinued RCTs were less likely to be published compared to completed trials. Reporting quality of trial protocols was not associated with premature discontinuation. In comparison to investigator-sponsored trials, industry-sponsored RCTs tend to perform better in prospectively registering trials, avoiding discontinuation due to poor recruitment, and making results available in trial registries. Missing data for planned sample size for 4 trial protocols was inserted from other sources (i.e., peer-reviewed publication; n = 3; trial registry; n = 1). b Split body (n = 2), parallel group with 2 consecutive randomizations (n = 1). Harm 6 (6%) 5 (9%) 1 (2%) 5 (83%) 6 (100%) 5 (83%) 0 (0%) Organizational/ strategic reasons 6 (6%) 6 (11%) 0 (0%) 3 (50%) 4 (67%) 1 (17%) 0 (0%) Two studies that stated slow recruitment as reason for discontinuation mentioned in addition another reason (i.e., organizational/strategic reasons n = 1; external evidence n = 1). b Counting the following reasons as not preventable: futility, harm, benefit, external evidence. Counting the following as preventable: poor recruitment, organizational/ strategic reasons, limited resources, and unclear reasons (assuming that discontinuation due to unclear reasons was mainly due to non-data-driven reasons [18]). RCT, randomized clinical trial. https://doi.org/10.1371/journal.pmed.1003980.t003

Comparison with other studies
A systematic review and meta-analysis published in 2018 found that in different medical specialties, 2% to 79% of RCTs were not registered [21]. The study authors highlighted that the proportion of registered trials increased over time [21]. However, they only considered published RCTs, whereas we had access to the trial protocols and were also able to explore if unpublished results were made available in a registry. Overall, 6% of the RCTs from our sample were not registered. When separately assessing published and nonpublished RCTs, these proportions were 2% and 21%, respectively, indicating that nonregistration is more common among nonpublished RCTs. The proportion of prospectively registered RCTs was 84%. Other studies show a wide range of lower proportions (24% to 72%) [9,22], depending on medical specialties, time frame assessed, and journals considered for selecting included RCTs. We can only speculate why we found a higher rate of prospectively registered RCTs. Reasons might be that there was a general improvement over the last years and that the included countries from which we selected RECs might be more stringent in enforcing registration prior to patient recruitment.
Compared to a decade earlier [14], the proportion of prematurely discontinued RCTs did not change. Further, discontinued RCTs were more likely to remain unpublished in both our prior [14] and current study, and the most common reason for discontinuation remained poor recruitment. Publication rates between our cohorts showed improvement, with 59% of approved trials appearing in a peer-reviewed journal in our prior study [14] and 79% in our current analysis. When assessing the availability of results either as a publication or in clinical trial registry, 87% of study results were available (not assessed for RCTs approved in 2000 to 2003 [14]). In addition, both studies found that industry-sponsored trials published their results more frequently and were associated with lower rates of discontinued trials [14]. A 2018 systematic review concluded that industry-sponsored trial publications were more comprehensively reported than investigator-sponsored [23]. In line with these results, we found that the reporting quality in RCT study protocols approved in 2012 was better for industrysponsored trials compared to investigator-sponsored RCTs. A systematic review by Schmucker and colleagues [24] revealed that 2 previous studies also assessed the publication rate of ethically approved RCT protocols. Both found low publication rates for protocols that were approved between 1988 and 1998 (i.e., 52%; 233/451; approved 1988 to 1998 in Switzerland [25]; 37%; 102/274 approved 1994 to 1995 in Denmark [26]). These findings are in line with our repeated metaresearch analysis, indicating that the publication rate has improved over the last decades.

Strengths and limitations
Strengths of our study include full access to the protocols of all trials approved by the collaborating RECs during the study period. We recruited reviewers with training in healthcare methodology to complete all data abstraction and considered only a limited number of variables in our regression models to reduce the chance of spurious associations. Further, we sampled trial protocols from the same RECs in both 2000 to 2003 and 2012 (with the exception of the added UK REC in 2012), which provides greater confidence in the shifts we observed regarding increased rates of RCT discontinuation and improved rates of publication over the past decade.
Our study has the following limitations: First, our sample size was modest, which may have limited the ability of our regression models to identify significant associations between protocol features and discontinuation or nonpublication. Second, of the 19 RCTs we classified as unregistered, the completion status of 11 was unclear. Thus, it is possible that some of those RCTs were never started and therefore the proportion of 6% unregistered RCTs may be smaller. Third, the SPIRIT checklist was created as a reporting guideline and not as a measurement tool for reporting quality [27]. However, we carefully operationalized the SPIRIT checklist and conducted various sensitivity analyses before using reporting quality estimates for the present study [15,17]. Fourth, we used trial protocols approved by RECs in Switzerland, the UK, Germany, and Canada, and the generalizability of our findings to RCT protocols approved by other RECs in these or other countries is uncertain. Fifth, regulatory aspects might have changed in some countries since 2012 (e.g., registration has been mandatory by law since 2014 in Switzerland, which had the highest proportion of unregistered trials [28,29]), hence it is possible that registration rates are higher nowadays.

Implications
Our study revealed encouraging results in terms of registration rates and making trial results available, but further efforts are still needed. Meerpohl and colleagues have developed 47 recommendations targeted at a variety of stakeholders [30]. Among others, they strongly recommend that legislators make trial registration mandatory, funding agencies request dissemination of all funded projects, and that RECs require trial registration before the recruitment of the first patient and request annual reports describing the dissemination of study results. Furthermore, publishing journals should remove barriers to publish negative or inconclusive results (e.g., from discontinued trials) and trial investigators should consequently make results available in trial registries [30]. Currently few investigator-sponsored RCTs make their results available in trial registries. The advantage of results posted in a trial registry over results reported in a published article may be the avoidance of spin [31]. Future research should address the current hurdles that exist among investigators to share study results in trial registries and how sharing results in registries could be promoted.
Other areas, such as discontinuation of RCTs due to preventable reasons and retrospective registration, need to be addressed too. Future research should assess if the rate of discontinued trials due to poor recruitment can be reduced with pilot or feasibility studies [18]. As stated by clinical trial registry representatives, trial registration and prospective trial registration should be enforced by publishing journals (including checking if trial registration exists) [32]. In case a trial was not prospectively registered, authors should at least explain in the published article why this was not done.

Conclusions
In our sample of RCTs approved by RECs from 4 countries, almost all were registered; however, 1 in 10 trials was registered retrospectively, which could result in methods being altered by study findings (e.g., changing the primary outcome [33]). Furthermore, 1 in 5 unpublished trials were not registered, and only 1 in 6 investigator-sponsored trials made results available in a trial registry. Higher reporting quality of trial protocols was positively associated with peer-reviewed publication, but not with prevention of trial discontinuation, highlighting the importance of feasibility assessments before embarking on a definitive trial. Despite a decade of efforts, premature trial discontinuation and nonpublication of RCTs remain common and comprise important targets to reduce waste in research.  Table. Factors associated with (a) publishing main results in a peer-reviewed journal; (b) discontinuation of trials due to poor recruitment; and (c) discontinuation of trials due to preventable reasons. UK samples excluded because variable "CTU/CRO support" was not assessed. (DOCX)